The cave is located on a rocky cliff overlooking a body of water